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DETAILED ACTION 



Response to Amendment 

1 . Applicant's amendment filed on December 08, 2006 has been considered and entered in full. 

2. Applicant's amendments to the claims have been considered but are moot in view of the 
rejection(s) made below. 

Claim Rejections - 35 USC § 112 

3. The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of making and 
using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or 
with which it is most nearly connected, to make and use the same and shall set forth the best mode contemplated by 
the inventor of carrying out his invention. 

/ 

4. Claims 33-48 are rejected under 35 U.S.C. 112, first paragraph, as failing to comply with the 
written description requirement. The claim(s) contains subject matter, which was not described in 
the specification in such a way as to reasonably convey to one skilled in the relevant art that the 
inventor (s), at the time the application was filed, had possession of the claimed invention. 

The amendment filed December 12, 2006 presents new claims, which recites the new 
limitations such as "acquiring data representing a grayscale image; adding data representing a textual 
annotation to said acquired grayscale image data; displaying an annotated grayscale image comprising 
said grayscale image with textual annotation overlaid thereon 5 ' in claim 33 and "acquiring data 
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representing a HSV color image; adding data representing a textual annotation to said acquired HSV 
color image data; displaying an annotated HSV color image comprising said HSV color image with 
textual annotation overlaid thereon" in claim 45. The claims and specification defining applicant's 
invention as originally filed does not provide any support for these limitations. Nowhere in the 
specification, the support for these limitations was found. However, examiner does acknowledge the 
section "background of invention" which recites "In many applications, such as medical diagnostic 
imaging, images are saved with annotations burnt in. The annotations are typically burnt in by 
overlaying an arbitrary intensity value of text on the image" (para. 0002) but no support of the 
process of adding annotations to the image has been found in the claims and specification defining 
applicant's invention as originally filed. The applicant or inventor of the instant application, at the 
time the application was filed, had no possession of the invention as cited above in the section 
"background of invention' and further does not show any support of the process of adding the 
annotations to the images in the specification, rather the specification only supports the 
enhancement of the images that were already annotated. 



Claim Rejections - 35 USC § 103 



5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness 
rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 
102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the 
subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary 
skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 
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6. Claim 33 is rejected under 35 U.S.C. 103(a) as being unpatentable over Ikeshoji et al., U.S. 
Patent No. 5,761,339, further in view of applicant's admitted prior art (here AAA). 

Regarding claims 33, Ikeshoji discloses acquiring an a grayscale image data in 
which textual annotations are present using a scanner and then storing such data on a recording 
medium or further displaying it on a display and further discloses that the present invention relates 
to an image processing method which is suitable for correction of photograph or a picture or a 
document of characters (col. 1, lines 7-13; col. 2, lines 43-50). Now as per the claims, the claim 
limitations "acquiring data representing a grayscale image; adding data representing a textual 
annotation to said acquired grayscale image data" discloses acquiring the grayscale image first and 
then adding the annotations to said acquired grayscale image, which results in an annotated grayscale 
image. Ikeshoji's invention as discussed in the previous rejections as well as described here in this 
rejection is directed to enhancing or correcting images that are annotated grayscale images. Ikeshoji 
as cited above acquires an annotated grayscale image (which is manually or non-electronically 
annotated) but does not expressly teach the steps of electronically adding the annotations to the 
electronic grayscale image to obtain annotated grayscale image. However, it is very well known to 
generate annotated images and applicant has admitted in the background of the invention that such 
a process of generating an annotated image is very well known and is used in many application such 
as medical diagnostic imaging (AAA teaches, "In many applications, such as medical diagnostic 
imaging, images are saved with annotations burnt in. The annotations are typically burnt in by 
overlaying an arbitrary intensity value of text on the image" (para. 0002)). As well known in the 
medical diagnostic imaging patient's name along with other information is added to the image itself 
for the purpose of image labeling for the identification purposes. Further for the sake of arguments, 
examiner directs applicant to softwares like Adobe Photoshop and Microsoft paint which are very 
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well-known to be used for image editing, including annotation. Ikeshoji teaches generating a 
annotated image where the annotations are added to the paper and then the paper is scanned to 
generate an annotated image and AAA on the other hand, teaches scanning the image and then 
electronically adding the annotations to obtain an annotated image. Since both ways of obtaining an 
annotated image is very well known to be as prior art and applicant itself has agreed to such 
methods, therefore, it would have been obvious for one of ordinary skill in the art at the time of 
invention was made to use any of the method defined by the prior arts and it would be obvious to 
one of ordinary skill in the art at the time of invention was made to use AAA's teachings in the 
invention of Ikeshoji because both references teach generating annotated images and AAA's 
teachings would prevent the original image to be permanently annotated and AAA presents the ease 
of annotating the images since no paper and pen is required, however in the case of Ikeshoji the 
original paper annotated image will retain the annotations permanently and everything is done 
manually. 

Ikeshoji discloses removing data representing said textual annotation from said stored data 
representing said annotated grayscale image to derive data representing an unannotated grayscale 
image (figure 1; element 20D being the first modified image, which is obtained after annotations 
(elements 30, 32 and 34) are removed; element 30D represents removed annotations where 
annotations being textual (characters) (See Title; figure 1 clearly says image 30D being character and 
figure image; col. 2, lines 36-42 discloses characters (textual annotations) are drawn on an image). 

Ikeshoji further discloses processing said data representing an unannotated grayscale image 
(20D) using an algorithm to derive data representing a processed grayscale image (col. 5, lines 1-5; 
removing stains or making the stain inconspicuous). 
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Ikeshoji further discloses merging said removed data representing said textual annotation 
and said data representing said processed grayscale image, said merged data representing an 
annotated processed grayscale image (figure 3, element 10-1 being the merged image). 

7. Claims 34-44 are rejected under 35 U.S.C. 103(a) as being unpatentable over Ikeshoji et al, 
U.S. Patent No. 5,761,339, further in view of applicant's admitted prior art, and further in view of 
Macleod et al., U.S. Patent No. 5,778,092, and further in view of Bloomberg, U.S. Patent No. 
5,065,437. 

Claim 34 recites "The method as recited in claim 33, wherein said removing step comprises 
the following: deriving data representing a first binary mask defining one or more image regions; and 
multiplying said data representing said first binary mask and said data representing said annotated 
grayscale image to derive said unannotated grayscale image". Ikeshoji clearly teaches in figure 1, 
deriving a first modified image (element 20 D) from which annotations have been removed with the 
help of filtering (or masking). Removing a part of images by using a mask is very well known in the 
process of image segmentation where processing such as convolution further involving multiplying 
mask bits with original image is used and is generally used in differentiating and removing an image 
part from the original image. Ikeshoji does teach of using a filter or mask but does not specifically 
teach deriving a binary mask, however, Macleod teaches this generally used well-known method of 
deriving a binary mask (col. 2, lines 1-13; col. 4, lines 65-68 through col. 5, lines 1-16 and lines 30-68; 
col. 8, lines 4-10 and lines 25-40). 

Further supporting, in lines 57-62 of col. 11, MacLeod discloses that U.S. Patent No. 5,065,437, is 
incorporated by reference, thereby, not requiring the motivation to combine the references. 
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Bloomberg. Bloomberg provides the support for subject matter in claim 2 in (figure lB 3 col. 2, lines 
20-68; col. 4, lines 1-68; col. 6, lines 10-40). Therefore, it would have been obvious for one of 
ordinary skill in the art at the time of invention was made to use combined teachings of MacLeod 
and Bloomberg in the invention combined of Ikeshoji and AAA because all references are directed 
to image segmentation and the combined MacLeod and Bloomberg's teachings provide the well- 
known teachings that are generally used in image segmentation, which would further provide a 
robust and computationally efficient technique for identifying and separating regions (See 
Bloomberg, col. 2, lines 5-15). 

Claim 35 recites "The method as recited in claim 34, wherein said merging step comprises 
the following: inverting said data representing said first binary mask to derive data representing a 
second binary mask defining one or more annotation regions; multiplying said data representing said 
second binary mask and said data representing said annotated grayscale image to derive data 
representing a modified image; and merging said data representing said modified image and said data 
representing said processed grayscale image to derive said data representing said annotated grayscale 
image". Ikeshoji clearly shows in figure 1, deriving images 20D and 30D, where in image 20D 
annotations are removed and in image 30D only annotations are present, therefore these 2 images 
are clearly subject to the apparent use of a mask that when inverted would provide segmentation for 
extracting one part or the other from the original image. Figure 3 clearly shows merging said second 
modified image (30) and said processed image (20) to derive said merged image. 

Claim 36 recites "The method as recited in claim 33, wherein the merged textual 
annotations occupy the same pixels in said annotated processed grayscale image that the removed 
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textual annotations originally occupied in said annotated grayscale image". Ikeshoji clearly shows in 
figure 3 the image 10-1 which has annotation at the same image position as shown in figure 1 image 
10 and further support can be found in (col 4, lines 6-12). 

Claim 37 recites "the method as recited in claim 33, wherein said removing step comprises 
morphology-based processing and thresholding". MacLeod discloses dilation and erosion 
(morphological operations) (col. 7) and further teaches thresholding (col. 5). 

Claim 38 recites "The method as recited in claim 33, wherein said removing step comprising 
the following: grayscale erosion of said data representing said annotated grayscale image using a 
structuring element to derive data representing an eroded grayscale image; thresholding said data 
representing said eroded grayscale image to derive data representing a first binary mask; dilation of 
said data representing said first binary mask using said structuring element to derive data 
representing a second binary mask defining one or more image regions; and multiplying said data 
representing said second binary mask and said data representing said annotated grayscale image to 
derive said data representing said image". The subject matter recited in the claim 38 is nothing but a 
morphological operation (opening: erosion followed by dilation), which is very well-known to be 
used in the process of identifying and separating image portions. MacLeod clearly teaches all the 
steps recited in claim 6 in (col. 11 through col. 12, lines 1-55). Bloomberg further provides the 
support in (col. 9, lines 45-68 through col. 10, lines 1-25). 
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Regarding claim 39, citing the well-known fact of modifying the image using an image 
mask, and in further view of Ikeshoji providing merging of images as explained before in tfie 
rejection of claim 3. Claim 7 has been similarly analyzed and rejected as per claims 35 and 38. 

Regarding claim 40, MacLeod discloses said removing step comprises thresholding and 
pixel connectivity-based analysis" (col. 6, lines 44-56). See Bloomberg (col. 2, lines 28-60; col. 4, lines 
40-65; col. 7, lines 1-65). 

Regarding claim 41, claim 41 additionally recites the limitation where "using 8-connected 
analysis to reject segments smaller than a prespecified size from said first binary mask to 
derive a second binary mask". MacLeod discloses of the same in (col. 6, lines 44-56; col. 10, lines 1- 
40). All other limitations have been similarly analyzed and rejected as per claims 33-40. 

Regarding claim 42, claim 42 has been similarly analyzed and rejected as per claims 41, 39 

and 35. 

Regarding claim 43, claim 43 in addition to claim 41 recites "removing holes from a 
second binary mask to derive a third binary mask". MacLeod teaches mask reconstruction for filling 
(removing) the interior holes (col. 11, lines 63-68 through col. 12, lines 1-15). 

Regarding claim 44, Ikeshoji discloses filtering to enhance said modified ultrasound image 
(col. 6, lines 10-20). 
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8. Claims 45-48 are rejected under 35 U.S.C. 103(a) as being unpatentable over Ikeshoji et al., 
U.S. Patent No. 5,761,339, further in view of applicant's admitted prior art, and further in view of 
Macleod et al., U.S. Patent No. 5,778,092, and further in view of Bloomberg, U.S. Patent No. 
5,065,437 and further in view of Gonzales et al., 1992, Book publication "Digital image processing". 

As discussed before in the rejection of claims 33-44, the combined invention of Ikeshoji, 
AAA, MacLeod and Bloomberg provides techniques that are used on grayscale images. Now the 
question is, can the same techniques be used on color images? The answer would be yes and further 
support is provided by Gonzales. Gonzales teaches "the HIS model is ideally suited for image 
enhancement, because the intensity component is decoupled from the color information in an 
image. Therefore any monochrome enhancement technique can be carried over as a tool for 
enhancing full-color images. It simply calls for converting the image to the HIS format, processing 
the intensity component, and converting the result to RGB for display" (page 247, 4 th paragraph) 
and apparendy the HS components ^ould have to be merged back in as the processing is done on 
color images and the output required would be a color image. Also, converting an RGB to HSI and 
HSI to RGB model is very well known and is further taught by Gonzales on pages 229 and 235. 
Therefore, providing a motivation for one of ordinary skill in the art at the time of invention was 
made to combine Gonzales's teachings in the combined invention of Ikeshoji, AAA, MacLeod and 
Bloomberg. All other limitations of claim 45 have been similarly analyzed and rejected as per claims 
33-44. 

Claims 46-48 have been similarly analyzed and rejected as per claims 34-45. 
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9. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office 
action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is 
reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE MONTHS 
from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the 
mailing date of this final action and the advisory action is not mailed until after the end of the 
THREE-MONTH shortened statutory period, then the shortened statutory period will expire on 
the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be 
calculated from the mailing date of the advisory action. In no event, however, will the statutory 
period for reply expire later than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Manav Seth whose telephone number is (571) 272-7456. The examiner can 
normally be reached on Monday to Friday from 8:30 am to 5:00 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, 
Bhavesh Mehta, can be reached on (571) 272-7453. The fax phone number for the organization 
where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR system, 
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see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, 
contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



Manav Seth 
Art Unit 2624 
February 16, 2007 
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